Annotation Tool

DRUID Annotation Tool helps bot authors leverage the NLU model of their chatbots based on real conversations with the users. This tool helps bot authors to use actual “real-life” bot conversation as data source for improving the bot’s NLU model, in a new single user-friendly interface.

Important!  This feature is available for DRUID Connectors v1.45 or above. If you have a hybrid or on premise DRUID deployment, install DRUID Connector Host version 1.45 or above.

The Annotation tool enables bot authors to:

  • View the chatbot conversation history in a specific date range.
  • Filter the conversation history based on annotation statuses and accuracy scores.
  • Get insights on the accuracy score for all messages in the conversation history.
  • Use messages from Conversation History as training phrases on desired flows (new or existing ones) and add them to train or test sets.

How to use the Annotation Tool

  1. Go to your DRUID production environment.
  2. Select the bot and solution and from the main menu, click Evaluation. Click the Conversation History tab.
  3. Filter the conversation history. DRUID provides you with several options to filter the conversation history. For more information, see Viewing Conversation History.
  4. At the top-right corner of the page, click the Download icon ().

  5. On your DRUID test environment, select the bot and solution and from the main menu, click Evaluation. In the Train Set tab, evaluate the existing general metrics of the bot training model.
    Note:  For backup purposes, we recommend you to download the chatbot train set.
  6. Click the Conversation History tab.
  7. Upload the conversation history file you downloaded from the production environment.

  8. Annotate the data based on your needs (add intents from conversation history as new training/test phrases). For more information on how to annotate messages in the Conversation History, see Viewing and Annotating.
  9. Click the Train Set tab and train the bot by clicking the Train button.
  10. Compare the general metrics of the bot training model with the ones prior the conversation history upload.
    11. Hint:   If the new metrics are worse than the previous ones, make the necessary corrections using DRUID’s available evaluation tools.
  11. Once the new metrics satisfy your needs, download the train set.
  12. Go back on the production environment and select your bot and solution, click Flows and upload the train set.

  13. Train the bot.

Related Topics Link IconRelated Topics